-
Couldn't load subscription status.
- Fork 22
[Enhancement] Ranking importance (prefer lower runtimes over smaller diff lines) #685
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
PR Reviewer Guide 🔍Here are some key observations to aid the review process:
|
PR Code Suggestions ✨Explore these optional code suggestions:
|
|
talk to @aseembits93 about this since there is some ideas on this problem |
|
closing in favor of #717 |
PR Type
Enhancement
This is still experimental, but the core idea is the
normalization: which is better than sorting in that case because if there is a very low runtime or a low diff it will standout
relativelyto other values.So suppose if we have these metrics
runtime = [60, 40, 4]
diffs = [10, 9, 10]
the total score we would have currently is
{0: 3, 1: 1, 2: 2}so it will pick the second candidate of 40 runtime and 9 lines diff (which is not the best here)but with normalization and weights the ranking dict would be
{0: 1.0, 1: 0.4821428571428572, 2: 0.25}so it would pick the 4 runtime with 10 lines of diffso it's all about which candidate is better related to other candidates
weights: for the 3 and 1 weights, it's for saying runtime is 3 times more important than diff, I still need to play with these two numbers to see what is the best percentage
Description
Add weight utilities for normalized importances
Implement metric normalization and scoring
Update
determine_best_candidatewith weightingRemove old rank summation approach
Diagram Walkthrough
File Walkthrough
code_utils.py
Introduce weighted metrics utility functionscodeflash/code_utils/code_utils.py
choose_weightsutility for normalizationnormalizefunction for metrics scalingcreate_score_dictionary_from_metricsfor scoringfunction_optimizer.py
Switch to weighted metric ranking logiccodeflash/optimization/function_optimizer.py
runtimes_listanddiff_lens_listcreate_rank_dictionary_compactlogic